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REMARKS 



THE CLAIMS 



REJECTION UNDER 35 U.S.C 103 



A. Claims 1. 3. 5. 6. 13. 16. 39. 40. 42. 45. and 46 

Claims 1, 3, 5, 6. 13, 16, 39, 40, 42, 45, and 46 were rejected under 35 U.S.C. 
103(a) as being unpatentable over Jain etal., U.S. Pat. No. 5,911,139 (hereinafter 
"Jain"), in view of Polnerow et al., U.S. Pat. No. 5.813,006 (hereinafter "Polnerow"). 

Applicant respectfully traverses this rejection and submits that the claims on file 
are not obvious in view of Jain in light of Polnerow, and are patentable thereover. In 
support of this position, Applicant submits the following arguments: 



B. Legal Standards for Obviousness 



The following are court opinions set the general standards in support of 
Applicant's position of non-obviousness, with emphasis added for clarity purpose: 



• " Obviousness cannot be established by combining the teachings of the prior art 
to produce the claimed invention, absent some teaching or suggestion 
supporting the combination." In re Fine, 837 F.2d at 1075, 5 USPQ2d at 1598 
(citing ACS Hosp. Sys. v. Montefiore Hosp., 732 F.2d 1572, 1577, 221 USPQ 
929, 933 (Fed. Cir. 1984)). Wiiat a reference teaches and whether it teaches 
toward or awav from the claimed invention are questions of fact. See Raytheon 
Co. V. Roper Corp., 724 F.2d 951, 960-61. 220 USPQ 592, 599-600 (Fed. Cir. 
1983), cert, denied, 469 U.S. 835, 83 L Ed. 2d 69, 105 S. Ct. 127 (1984). " 

• "When a rejection depends on a combination of prior art references, there must 
be some teaching, suggestion, or motivation to combine the references. See 
In re Geiger, 815 F.2d 686, 688. 2 USPQ2d 1276. 1278 (Fed. Cir. 1987)." 
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Obviousness r^n only be established bv nnmbininQ or modifying the teachings of 
the prior art to produce the claimed invention where there is some teaching. 
suggestion, or motivation t o do so found either explicitly or implicitly in the 
references themselves or in the knowledge generally available to one of ordinary 
skill In the art. See MPEP 2143.01; In re Kotzab. 217 F.3d 1365, 1370, 55 
USPQ2d 1313, 1317 (Fed. Cir. 2000); In re Fine. 837 F.2d 1071. 5 USPQ2d 1596 
(Fed. Cir. 1988); and In re Jones. 958 F.2d 347. 21 USPQ2d 1941 (Fed. Cir. 
1992). 

"With respect to core fectual findings in a determination of patentability, however, 
thfi Board cannot simpiv reach conclusions ba sed on its own understanding 
or experience - or on its assessment of what would be basic knowledge or 
common sense. Rather, the Board must p oint to some concrete evidence in 
the record in support of these findings." See In re Zurko, 258 F.3d 1379 (Fed. 
Cir. 2001). 

"We have noted that evidence of a suaoestion. teachin g- or motivation to 
combine may flow from the prior art references themselves, the knowledge of 
one of ordinary skill in the art, or, in some cases, from the nature of the problem to 
be solved, see Pro-Mold & Tool Co. v. Great Lakes Plastics, inc., 75 F.3d 1568. 
1573 37 USPQ2d 1626. 1630 (Fed. Cir. 1996), Para-Ordinance Mfg. v. SGS 
Imports intern., Inc., 73 F.3d 1085, 1088, 37 USPQ2d 1237, 1240 (Fed. Cir. 
1995) although "the suggestion more often comes from the teachings of the 
pertinent references," Rouffet, 149 F.3d at 1355. 47 USPQ2d at 1456. The range 
of sources available, however, does not diminish the requirement for actual 
evidence. That is, the showing must be c lear and particular. See, e.g., C.R. 
Bard. 157 F.3d at 1352. 48 USPQ2d at 1232. Broad conciusorv statements 
reaardino the teaching of multiple references , standing alone, are not 
"evidence." E.g.. McElmuny v. Arkansas Power & Light Co., 995 F.2d 1576, 
1578, 27 USPQ2d 1129, 1131 (Fed. Cir. 1993) ("Mere denials and conclusory 
statements, however, are not sufficient to establish a genuine issue of material 
fact."); In re Sichert. 566 F.2d 1154. 1164, 196 USPQ 209, 217 (CCPA 1977)." 
See In re Dembiczak, 175 F. 3d 994 (Fed. Cir. 1999). 

To prevent the use of hindsight based on the invention to defeat patentability of 
the invention, this court reouires the e xaminer to s >ft>w a motivation to 
combine the references that create the case of obviousness. In other words, the 
examiner must s^ow reasons that the skilled artisan, confronted with the same 
problems as the inventor and with no knowledge of the claimed invention, would 
select the elements from the cited prior art references for combination in the 
manner claimed ." See In re Rouffet, 149, F.3d 1350 (Fed. Cir. 1998). 

MPEP 21 43.01 -"The Prior Art Must Suggest The Desirability Of The Claimed 
Invention. There are three possible sources for a motivation to combine 
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references: the nature of the problem to be solved, the teachings of the prior art, 
and the knowledge of persons of ordinary skill in the art." In re Rouffet, 149 F.3d 
1350, 1357, 47 USPQ2d 1453, 1457-58 (Fed. CIr. 1998) (The combination of 
the, references taught every element of the claimed invention, however 
without a motivation to combine, a rejection based on a prima facie case of 
obvious was held improDer V The level of skill in the art cannot be relied upon 
to provide the suggestion to combine references. Al-Site Corp. v. VSI Intl Inc., 
174 F.3d 1308, 50 USPQ2d 1161 (Fed. Cir. 1999). 

• The mere fact that references can be combined or modified does not render the 
resultant combination obvious unless the prior art also suggests the 
desirability of the combination . In re Mills, 916 F.2d 680. 16 USPQ2d 1430 
(Fed. Cir. 1990). Although a prior art device "may be capable of being modified to 
nan the way the apparatus is claimed, there must be a suggestion or 
motivation in the reference to do so." 916 F.2d at 682, 16 USPQ2d at 1432,). 
See also In re Fritch, 972 F.2d 1260. 23 USPQ2d 1780 (Fed. Cir. 1992) (flexible 
landscape edging device which is conformable to a ground surface of varying 
slope not suggested by combination of prior art references). 

• If the proposed modification would render the prior art invention being 
modified unsatisfactory for its intended purpose, then there is no suggestion or 
motivation t o make the proposed modification. In re Gordon. 733 F.2d 900, 221 
USPQ 1125 (Fed. Cir. 1984). 

C. Jain 



Prior to discussing the substantive merit of the obviousness rejections, it might be 
desirable to summarily review the Jain patent. Jain generally describes a system and 
method for content-based s earch and retrieval of visual objects. A base visual 
information retrieval (VIR) engine utilizes a set of universal primitives to operate on 
the visual objects. An extensible VIR engine allows custom, modular primitives to be 
defined and registered. A custom primitive addresses domain specific problems can 
utilize image understanding techniques. Object attributes can be extracted over the 
entire image or over only a portion of the object. A schema is defined as a specific 
coilection of primitives . A specific schema implies a specific set of visual features 
to be processed and a conresponding feature vector to be used for content-based 
similarity scoring. A primitive registration interface registers custom primitives and 
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facilitates storing of an analysis function and a comparison function to a schema 
table. A heterogeneous comparison allows objects analyzed by different schemas to 
be compared if at least one primitive is in common between the schemas. A 
threshold-based comparison is utilized to improve performance of the VIR engine. A 
distance between two feature vectors is computed In any of the comparison 
processes so as to generate a similarity score. Refer to the Abstract. 

"The VIR Engine 120 comprises two main modules: an "Image Analysis'* module 
122 and an "Image Comparison" module 124. The image analysis module 122 
receives inputs from either module 1 08 or 11 0 to generate a query target or from the 
insertion module 112 for adding a new image into the database 132, The output of 
the image analysis module 122 is a feature vector (FV) that describes the visual 
object passed to it by one of modules 108, 1 10 or 1 1 2, The FV is passed on to the 
database engine 130. In addition, if module 112 was used to insert the image into the 
database, both the FV for the Image and the image itself are stored in the database 
132 (as seen in FIG. 5B). The image analysis module 122 will be described in greater 
detail hereinbelow. 

The image comparison module 124 receives a query target FV and a FV for the 
image being tested or compared from the database engine 130. The output of the 
Image comparison module 124 is a similarity score that is sent to a "Ranked List 
Management" module 134. A plurality of images from the database 132 are 
compared one at a time to the query image by the image comparison module 124. 
The resultant similarity scores are accumulated by the module 134 so as to provide a 
ranl^ in an order of their similarity to the query image. The ranl^ed results of the list 
management module 134 are provided to a 'Thumbnail Results Browser" 136 for 
display to the user 102 through the computer I/O 104, An exemplary screen display 
of ranked results is shown in FIG. 4." Reference is made to FIG. 1 A, and further to 
column 9, lines 41 - 67. 
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" What IS needed is a new multimedia infomiation system technology model such 
as a visual infomiation management system (VIMSYS) model. Unlike traditional 
database systems, this model recognizes that most users prefer to search image 
and video information bv what the image or video actually contains, rather than 
by keywords or descriptions associated with the visual information . The only 
proper method by which the user can get access to the content of the image Is by 
using image-analysis technology to extract the content from an image or video . 
Once extracted, the content represents most of what the user needs in order to 
organize, search, and locate necessary visual information. 

This breakthrough concept of content extraction alleviates several technological 
problems. The foremost benefit is that it gives a user the power to retrieve visual 
information bv asking a guerv like "Give me ail pictures that loolc like this," The 
system satisfies the query by comparing the content of the auerv Picture w ith that 
of all target pictures in the database. This is called Query By Pictorial Example 
(QBPE), and is a simple form of content-based retrieval , a new paradigm in 
database management systems." Reference is mad to column 2, lines 53 - 65, with 
emphasis added. 

D. Application of the Obviousness Standard to the Present Invention 

D.1 . Elements not disclosed bv Jain 

Applicant respectfully submits that the following elements and combination of 
elements and resulting features recited in the representative independent claim 1 and 
the claims dependent thereon, are not disclosed In Jain (with emphasis added): 

"1 . A system for automatically associating contextual input data w ith 
available multimedia resources, comprising: 

a contextual input device for capturing the contextual input data : 
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an assistant device for processing the contextual input data captured 
by the contextual input device, and for formulating a query based on 
processed contextual input data and a user profile : and 

a contextual multimedia association module for associatino the 
processed contextual input data w itli tlie multimedia resources and for 
generating association matches.'' Emphasis added. 

D.2. Summary of Arguments 

Applicant incorporates by reference all the arguments made in the previous 
response, and further respectfully submits a summary of the arguments to be 
developed in more detail in subsequent sections: 

D.2.a: Jain does not capture contextual input data. 

D.2,b: Jain does not formulate a auerv based on the processed contextual input 
data and the user profile. 

D,2-a! Jain does not capture contextual input data 

As it is clear from the Jain excerpts above, Jaln^s method is strictly content- 
based, in that It is limiting to discovering and cataloguing the content of the Input 
image . Jain does not capture contextual input data , as understood and defined in 
the present application. 

The term "contextualMs a l^ev concept to the present invention, and is mentioned 
eight times in the instant claim 1 . Applicant respectfully requests that this term be 
given its proper weight and not be summarily dismissed. The term "contextual" is 
clearly defined in the present application as follows: 

AM9'-99-0227 13 
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"As used herein, "contextual" means or implies the surrounding 
circumstances in which an object exists or an event occurs . For example. 
the contextual content of a photograph can be all the information surroundino 
the situation in which the photograph is taken, includina soedal and physical 
parameters such as time, location, elevation, etc. , as well as infonnation 
gathered by various sensors such as temperature, pressure, humlditv. light 
level, sound, and acceleration sensors, and user interface elements such as 
buttons, switches, etc. T he contextual data helps to understand the context of 
the acquired data." Reference is made to page 7, lines 5 - 12 of the present 
application, with emphasis added for clarity. 

" Exemplary, non-exclusive contextual data include environmental and 
Qeospatial coordinates, time, temperature, location, speed, motion, 
acceleration, and other parameters ." Reference is made to page 7, lines 1 - 3 
of the present application, with emphasis added for clarity. 

The concept of capturing contextual input contextual input data, and then 
processing the captured contextual input data is neither taught nor suggested in Jain. 

The text cited by the Examiner, which was reproduced in part, above, supports 
the finding that the capture of visual information is limited in scope to the content of 
the captured data. THE CITED TEXT CONFiRIMS THAT JAIN IS CONTENT- 
BASED. AS OPPOSED TO CONTEXT-BASED , in that Jain seems to disregard the 
surrounding circumstances, such as the environmental and geospatial 
coordinates, time, temperature, location, speed, motion, or acceleration in which the 
content exists or the event occurs . 
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D. 2,b. Jain does not teach automatically 

formulating the query based on the 
contextual input data and the user profile 



M ) Applicant aorees with the Examiner t hat Jain does not teach automaticaily 
formulating the query based on the user proflle. 



(2) Furthermore. Applicant submits that Jain does not teach automatically 
formulatirtq the ouerv based on the contextual Input data , since, as stated earlier, 
Jain seems to ignore the contextual data, and rather focuses on the content data. 



In order to compensate for the first missing element, the office action resorts to 
Polnerow, as discussed below. 



D. 3. The combination of Jarn and Polnerow is improper 



In support of the combination of Jain and Polnerow, the office action adds that 
Polnerow teaches searching based on user profile (see col. 1, lines 55-65 and 
column 10, lines 53-54), reasoning that: 



"It would have been obvious to a person of ordinary skill in the art at the 
time the invention was made to employ the teachings of Polnerow within the 
system of Jain by automatically searching database for information based on 
user profile within the contextual input data and multimedia data system 
associating system and method because. Jain teaches of user specific 
property or attribute of the image f or performing queries (see col.4, lines 21- 
40) and such attributes could be users preference accx>rding to the users 
profile. Since user supplies the property or attributes, theses attributes or 
properties could be derived according to users preference such as users 
profile ." 
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Applicant respectfully traverses this rejection ground and submits ttie following 
arguments: 

(1) Jain limits the "specific property or attribute of the Image" to what is refen-ed to as 
"primitives" of the content. See for example, column 6, lines 20-21 in Jain. There is, 
however, no justification or teaching in Jain to expand the "specific property or 
attribute of the Image" to cover "the user's profile," particularly that Jain does not 
consider the user's profile to be an attribute of the image. This latter point is 
discussed In more detail below. 

(2) In addition, Poinerow does not disclose fomiuiating the query based on the user's 
profile ' Tor automaticallv afi&oclatina contextual input data w ith available 
multimedia resources." 

(3) Furthermore, neither Poinerow nor Jain teaches or suggests Ibrmulating a query 
based on processed contextual input data ." as presented eariier, and as clearty 
defined In the present application. 

(4) The Examiner bases the ground for combining Jain on Poinerow on the user's 
profile, but does not provide any ground for combining these two references based 
on the processed contextual input data . Applicant submits that these two 
references cannot be justifiably combined based on the processed contextual input 
data, since this feature is neither suggested nor taught by any of these two 
references. 

Claim 1 and the claims dependent thereon are allowable, and such allowance is 
respectfully requested. Similariy, independent claims 13 and 39 and the claims 
dependent thereon are allowable for reciting comparable elements to those of claim 
1. 
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E. Dependent Claims 

E. 1. Claims 7. 17. and 43 

Claims 7, 17, and 43 were rejected under 35 U.S.C. 103(a) as being unpatentable 
over Jain in view of Bull. 

Applicant respectfully traverses this rejection and submits that the rejected claims 
are not obvious in view of Jain and Bull, and are thus patentable thereover. In 
support of this position, Applicant submits Bull was referred to as teaching the 
development of a user profile. 

Applicant respectfully submits that the same arguments presented earlier in favor 
of allowance of claim 1 over Jain and Polnerow are equally applicable to the 
combination of Jain and Bull. Further, Bull stands for the generally known concept of 
developing a user profile, which, by itself, does not constitute the novel and 
patentable aspect of the present invention. 

The combination of Jain and Bull does not disclose the following elements of the 
representative claim 7: 

• "formulating a query leased on processed contextual input data and a user 
profile"; and 

• "the assistant device develops the user profile based on association matches 
that were previously presented to the user". 

Claim 7 and the claims dependent thereon are allowable, and such allowance is 
respectfully requested. Similarly, independent claims 17 and 43 and the claims 
dependent thereon are allowable for reciting comparable elements to those of claim 
7. 



AM9-99-0227 17 



Received froin<4083235112>at6/15f0311:13:28PM [Eastern Daylight Time] 



06/15/03 SUN 20:12 FAX 40832 



@019 



E.2. Claims 8. 18. and 44 

The office action states that with regard to claims 8, 18, and 44, "Jain does not 
teach wherein the a ssistant device updates the user digital profile based on 
recent ass ociation matches . Bull teaches of uodatina the user digital profile based 
on recent association matches fsefi coi.8, line 65 to coi.9. line 2 and col.12, lines 2- 
4). It would have been obvious to a person of ordinary sl<ili in the art at the time the 
invention was made to employ the teachings of Bull within the system of Jain by 
updating user profile within the contextual input data and multimedia data system 
associating system and method because, this would i<eep tracl< of the potential future 
user as well as i<eep track of the user's preferences for future search "primitives" to 
speed up the processing time." Emphasis added. 

Applicant respectfully submits that the following elements and combination of 
elements and resulting features recited in the representative dependent claim 8 and 
the claims dependent thereon, are not disclosed in Jain (with emphasis added): 

"8. The system according to claim 7, wherein the assistant device updates 
the user digital profile based on recent association matches ." Emphasis 
added 

Applicant incorporates herein the presentation made earlier in favor of the 
allowance of claim 7. and further respectfully submits that claim 8 depends on claims 
1 and 7 as now amended, to include the following elements that are missing from 
Jain and Bull: 

* formulating a query based on processed contextual Input data and a user 



• "the assistant device develops the user profile based on association matches 
that were previously presented to the user"; and 



profile"; 
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• "the assistant device updates the user digital profile based on recent 
association matches". 

Claim 8 and the claims dependent thereon are allowable, and such allowance is 
respectfully requested. Similarly, independent claims 18 and 44 and the claims 
dependent thereon are allowable for reciting comparable elements to those of claim 
8. 

E, 3. Remaining dependent claims 

The remaining dependent claims are allowable for depending on the allowable 
claims 1, 13, and 39, as presented earlier. 

TELEPHONE INTERVIEW 

The undersigned respectfully reiterates his request for a telephone Interview with 
the following agenda: "Discuss claim 1 in view of Jain, Polnerow. and Bull," 
particularly in view of the "contextual" input data concept that is not disclosed in any 
of these three references. 



AM9-99-0227 19 



Received from < 4083235112 > at 6/1S/03 11:13:28 PM [Eastern Daylight Time] 



06/15/03 SUN 20:12 FAX 40832 




[g|021 




CONCLUSION 



All the claims presently on file in the present application are in condition for 
immediate allowance, and such action is respectfully requested. If it is felt for any 
reason that direct communication would serve to advance prosecution of this case to 
finality, the Examiner is Invited to call the undersigned at the below-listed telephone 
number. 



Respectfully submitted, 




Date: June 16. 2003 



Samuel A. Kassatly 
Atlomey for Applicant 
Reg. No. 32,247 
Tel. (408) 323-5111 
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